Des illusions aux hallucinations visuelles

Des illusions aux hallucinations visuelles:
une porte sur la perception

Laurent Perrinet

JNLF 2019, 18/4/2019

Acknowledgements:

Rick Adams and Karl Friston @ UCL - Wellcome Trust Centre for Neuroimaging
Jean-Bernard Damasse, Laurent Madelain and Anna Montagnini - ANR REM
Frédéric Chavane - INT

This work was supported by ANR project "Horizontal-V1" N° ANR-17-CE37-0006.

Des illusions aux hallucinations visuelles:
une porte sur la perception

Laurent Perrinet

JNLF 2019, 18/4/2019

Acknowledgements:

Rick Adams and Karl Friston @ UCL - Wellcome Trust Centre for Neuroimaging
Jean-Bernard Damasse, Laurent Madelain and Anna Montagnini - ANR REM
Frédéric Chavane - INT

This work was supported by ANR project "Horizontal-V1" N° ANR-17-CE37-0006.

Hering illusion

Akiyoshi KITAOKA

https://www.illusionsindex.org/i/rotating-snakes

See also Professor Kitaoka’s personal website at http://www.ritsumei.ac.jp/~akitaoka/index-e.html

The image consists of an arrangement of snake-like concentric circles, defined by contrasting regions of colour.

Instructions

Allow your gaze to move naturally over the figure, coming to rest from time to time. Try fixing your gaze at a point and see what happens. Effect

You should have a visual experience as of the 'snakes' rotating, when in fact they are stationary. When you fix your gaze this illusory motion ceases. Note that some snakes rotate clockwise, others anticlockwise. The Rotating Snakes Illusion evokes a perceptual experience of illusory motion. It was invented by Japanese psychologist and academic Akiyoshi Kitaoka (Kitaoka and Ashida 2003). It is one of a class of peripheral drift illusions; whatever part of the figure is in the centre of our visual field appears motionless (as indeed it is), while the parts seen in our peripheral vision appear to move.

No-one is quite sure as to how the Rotating Snakes illusion works, although there is some consensus that it involves a difference in the processing latency of signals corresponding to different parts of the figure. Conway et al. (2003) propose that high-contrast areas are processed faster than low-contrast areas, where contrast is defined globally over the entire receptive field of an individual retinal neuron; in this case, the regions of highest contrast appear in the outermost ‘coil’ of the snakes. The illusory motion is then explained as an example of the reverse phi phenomenon first described in Anstis and Rogers (1974): a bright spot appearing and fading at some point in the visual field, subsequently followed by a dark spot appearing and fading at some other point, will create a sense of motion from the dark stimulus toward the light stimulus if this pattern is cycled. Looking closely at the figure, you will notice that adjacent ‘snakes’ are patterned so that the colours appear in the opposite order. If the reverse phi phenomenon explains illusory motion, the reversing patterns explain why some snakes rotate clockwise and others anticlockwise. A processing latency mechanism is consistent with the fact that the illusion ceases when our gaze is fixed, because in that case the signal from each part of the visual field is fairly constant. This suggests that blinks and small involuntary movements of the eye called ‘saccades’ may play an important role in triggering the illusion (Otero-Mill et al. 2012).

The Rotating Snakes Illusion is also interesting because it is relevant to debates about modularity, cognitive penetration, and the nature of experience. To explain: on the hypothesis that the mind is modular, a mental module is a kind of semi-independent department of the mind which deals with particular types of inputs, and gives particular types of outputs, and whose inner workings are not accessible to the conscious awareness of the person – all one can get access to are the relevant outputs. So, in the case of the Rotating Snakes Illusion, a standard way of explaining why experience of the illusion persists even though one knows that one is experiencing an illusion is that the module, or modules, which constitute the visual system are ‘cognitively impenetrable’ to some degree – i.e. their inner workings and outputs cannot be influenced by conscious awareness. For a general discussion of cognitive penetration, see Macpherson (2012).

Philosophers have also been interested in what illusions like the Rotating Snakes Illusion can tell us about the nature of experience. For example, in the case of experiencing the Rotating Snakes Illusion, it would seem to be that the one can know that nothing on the screen rotates whilst at the same time experiences rotating snakes. If so, then this might count against the claim the perceptual states are belief-like, because if perceptual states were belief like then, when experiencing the Rotating Snakes Illusion one would simultaneously believe that the snakes were, and were not, rotating. This would seem to entail that one was being irrational, because one would simultaneously be holding contradictory beliefs. But it seems highly implausible that one is being irrational when under going this illusion. For discussion of this general point about whether perceptions are like beliefs, see Crane & French (2016).

http://i2.cdn.cnn.com/cnnnext/dam/assets/150410134301-cat-going-up-or-down-super-169.jpg

#TheDress

https://en.wikipedia.org/wiki/The_dress

The dress is a photograph that became a viral internet sensation on 26 February 2015, when viewers disagreed over whether the dress pictured was coloured blue and black, or white and gold. The phenomenon revealed differences in human colour perception, which have been the subject of ongoing scientific investigations into neuroscience and vision science, with a number of papers published in peer-reviewed science journals.

The photo originated from a washed-out colour photograph of a dress posted on the social networking service Tumblr. Within the first week after the surfacing of the image, more than 10 million tweets mentioned the dress, using hashtags such as #thedress, #whiteandgold, and #blackandblue. Although the actual colour was eventually confirmed as blue and black,[3][4] the image prompted many discussions, with users debating their opinions on the colour and how they perceived the dress in the photograph as a certain colour. Members of the scientific community began to investigate the photo for fresh insights into human color vision.

The dress itself, which was identified as a product of the retailer Roman Originals, experienced a major surge in sales as a result of the incident. The retailer also produced a one-off version of the dress in white and gold as a charity campaign.

Two ways in which the photograph of The dress may be perceived: * black and blue under a yellow-tinted illumination (left figure) or * white and gold under a blue-tinted illumination (right figure).

Paréidolie

Cydonia Mensae (1976) Viking Orbiter image

Cydonia was first imaged in detail by the Viking 1 and Viking 2 orbiters. Eighteen images of the Cydonia region were taken by the orbiters, of which seven have resolutions better than 250 m/pixel (820 ft/pixel). The other eleven images have resolutions that are worse than 550 m/pixel (1800 ft/pixel) and are of limited use for studying surface features. Of the seven good images, the lighting and time at which two pairs of images were taken are so close as to reduce the number to five distinct images. The Mission to Mars: Viking Orbiter Images of Mars CD-ROM set image numbers for these are: 035A72 (VO-1010), 070A13 (VO-1011), 561A25 (VO-1021), 673B54 & 673B56 (VO-1063), and 753A33 & 753A34 (VO-1028).[11][12]

In one of the images taken by Viking 1 on July 25, 1976, a two-kilometre-long (1.2 mi) Cydonian mesa, situated at 40.75° north latitude and 9.46° west longitude,[13] had the appearance of a humanoid face. When the image was originally acquired, Viking chief scientist Gerry Soffen dismissed the "Face on Mars" in image 035A72[14] as a "trick of light and shadow".[15][16] However, a second image, 070A13, also shows the "face", and was acquired 35 Viking orbits later at a different sun-angle from the 035A72 image. This latter discovery was made independently by Vincent DiPietro and Gregory Molenaar, two computer engineers at NASA's Goddard Space Flight Center. DiPietro and Molenaar discovered the two misfiled images, Viking frames 035A72 and 070A13, while searching through NASA archives.[17]
2.1 Later imagery

More than 20 years after the Viking 1 images were taken, a succession of spacecraft visited Mars and made new observations of the Cydonia region. These spacecraft have included NASA's Mars Global Surveyor (1997–2006) and Mars Reconnaissance Orbiter (2006–),[18] and the European Space Agency's Mars Express probe (2003–).[19] In contrast to the relatively low resolution of the Viking images of Cydonia, these new platforms afford much improved resolution. For instance, the Mars Express images are at a resolution of 14 m/pixel (46 ft/pixel) or better. By combining data from the High Resolution Stereo Camera (HRSC) on the Mars Express probe and the Mars Orbiter Camera (MOC) on board NASA's Mars Global Surveyor it has been possible to create a three-dimensional representation of the "Face on Mars".[20]

Outline

Illusions visuelles & hallucinations

Hallucinations: quand les illusions deviennenty réelles

Une neuro-anatomie fonctionnelle des illusions?

and at this instant, the sensed physical position is lagging behind (as represented here by $ au_s \cdot v 1~m$ ), that is, approximately at $45$ degrees of eccentricity (red dotted line),
while the position at the moment of emitting the motor command will be $.8~m$ ahead of its present physical position ($ au_m \cdot v$).
As a consequence, note that the player's gaze is directed to the ball at its present position (red line), in anticipatory fashion. Optimal control directs action (future motion of the eye) to the expected position (red dashed line) of the ball in the future --- and the racket (black dashed line) to the expected position of the ball when motor commands reach the periphery (muscles). This is obviously an interesting challenge for modelling an optimal control theory.

Khoei, Masson and LP (2017) PLoS CB

so let's go back on earth

... but first let now apply this model compensating for the aforementioned visual delays using a well described visual illusion: the flash-lag effect:

a first stimulus moves continuously across the screen along the central horizontal axis. In the FLE, as this moving stimulus reaches the center of the screen, a second stimulus is flashed just above it and in perfect vertical alignment. Despite the fact that the respective horizontal positions of each stimulus are physically identical when the flash occurs, the moving stimulus is most often perceived ahead of the flashed one.

debate for 80 years revived recently

motion extrapolation
differential latency
post-diction

we propose to extend the hypothesis previously proposed by Nihjawan that this effect is caused by the extrapolation of the stimulus' motion to compensate for the neural delay. However, this hypothesis was challenged by other hypothesis that this effect is due to either anatomy (differential latencies) or to the way visual awareness processes the sequence of events (the post-diction from Eagleman)

As a matter of fact, the motion extrapolation hypothesis was challenged because you can notice that the FLE is still present at initiation of the movement but this effect is not seen if the moving dot abruptly stops at the moment of the flash

Hallucinations: quand les illusions deviennenty réelles

Khoei, Masson and LP (2017) PLoS CB

For that, we replot the movies I have just shown by showing for the dot the Histogram of the estimated positions as a function of time for the source layer (Left) and the target layer (right). The left-hand column illustrates the predictive model before delay compensation. The right-hand column illustrates the motion extrapolation model with delay compensation. Histograms of the inferred horizontal positions (blueish bottom panel) and horizontal speed (redish top panel) are shown in columns as a function of time. A darker level corresponds to a higher probability, while a light color corresponds to an unlikely estimation. In particular, we focus on three particular epochs along the trajectory, corresponding to the standard, flash initiated and terminated cycles. The timing of these epochs flashes are indicated by dashed vertical lines. In dark, the physical time and in green the delayed input knowing $ au=100~ms$.
Activity in both models shows three different phases. First, there is a rapid build-up of the precision of the target after the first appearance of the moving dot (at $t=300~ms$). Consistently with the Frölich effect, the beginning of the trajectory is seen ahead of its physical position. During the second phase, the moving dot is correctly tracked as both its velocity and position are correctly inferred. In the source layer, there is no extrapolation and the trajectory follows the delayed trajectory of the dot (green dotted line). In the target layer, motion extrapolation correctly predicts the position at the present time and the position follows the actual physical position of the dot (black dotted line). Finally, the third phase corresponds to motion termination. The moving dot disappears and the corresponding activity vanishes in the source layer at $t=900~ms$. However, between $t=800~ms$ and $t=900~ms$, the dot position was extrapolated and predicted ahead of the terminal position. At $t=900~ms$, while motion information is absent, the position information is still transiently consistent and extrapolated using a broad, centered prior distribution of speeds. Although it is less precise, this position of the dot at flash termination is therefore not perceived as leading the flash.
Interestingly, and thanks to the reviewers of the paper, we could extend our results to the estimation of the dot position from the dMBP model during the motion reversal experiment. In the motion reversal experiment, the moving dot reverses its direction at the middle of the trajectory (i.e., at $t=500~ms$, as indicated by the mid-point vertical dashed line). In the left column (target layer) and as in previous slide, we show the histogram of inferred positions during the dot motion and a trace of its position with the highest probability as a function of time. As expected, results are identical to that in the previous slide in the first half period. At the moment of the motion reversal, the model output is consistent with previous psychophysical reports. First, the estimated position follows the extrapolated trajectory until the (delayed) sensory information about the motion reversal reaches the system (at $t=600~ms$, green vertical dashed line). Then, the velocity is quickly reseted and converges to the new (reversed) motion such that the estimated position ``jumps'' to a position corresponding to the updated velocity. In the right column (smoothed layer), we show the results of the same data after a smoothing operation of $ au_s=100~ms$ in subjective time. This different read-out from the inferred positions corresponds to the behavioral results obtained in some experiments, such as that from~\citet{Whitney98}.

Friston , Adams, LP and Breakspear (2012) Frontiers in Psychology

Adams, LP and Friston (2012) PLoS ONE

LP, Adams and Friston (2015) Biological Cybernetics

This schematic shows the dependencies among various quantities modelling exchanges of an agent with the environment. It shows the states of the environment and the system in terms of a probabilistic dependency graph, where connections denote directed (causal) dependencies. The quantities are described within the nodes of this graph -- with exemplar forms for their dependencies on other variables.
Hidden (external) and internal states of the agent are separated by action and sensory states. Both action and internal states -- encoding a conditional probability density function over hidden states -- minimise free energy. Note that hidden states in the real world and the form of their dynamics can be different from that assumed by the generative model; (this is why hidden states are in bold. )

Friston , Adams, LP and Breakspear (2012) Frontiers in Psychology

Adams, LP and Friston (2012) PLoS ONE

LP, Adams and Friston (2015) Biological Cybernetics

Active inference uses a generalisation of Kalman filtering to provide Bayes optimal estimates of hidden states and action in generalized coordinates of motion. As we have seen previously, the central nervous system has to contend with axonal delays, both at the sensory and the motor levels. Representing hidden states in generalized coordinates provides a simple way of compensating for both these delays.
This mathematical framework can be mapped to the anatomy of the visual system. Similar to the sketch that we have shown above, "compiling" (that is, solving) the equations of Free-energy minimization forms a set of coupled differential equations which correpond to different node along the visuo-oculomotor pathways.

Practically, the predictive power of AI in modeling such an agent is revealed by studying deviations from the median behavior within a population of agents. For instance, there are acute differences in the smooth pursuit eye movements (SPEM) between patients from (control) neurotypic or schizophrenic groups. First, SPEM are distinct from the saccades defined above as they are voluntary eye movements which aim at stabilizing the retinal image of a smoothly moving visual object. For a target following the motion of a pendulum for instance, the eye will produce a prototypical response to follow this predictable target. Interestingly, schizophrenic agents tend to produce a different pattern of SPEM in the case that the pendulum is occluded on half cycles (for instance, as it passes behind an opaque cardboard on one side from the midline). In general, SPEM may still follow the target, as it is occluded (behind the cardboard) yet with a lower gain [@Barnes91]. As the target reappears from behind the occluder, schizophrenic agents engage more quickly to a SPEM response [@Avila06]. Extending the agent modeled in [@Friston12], an agent which has the capability to smoothly follow such moving object was modeled in [@Adams12]. This model allows in particular to understand most prototypical SPEM as a Bayes-optimal solution to minimize surprise in the perception / action loop implemented in the agent's dependency graph.

Especially, by manipulating the a priori precision of internal beliefs at the different levels of the hierarchical model, one could reproduce different classes of SPEM behaviors which reproduce classical psychophysical stimuli. For instance, [@Adams12] found for the half-cycle occluded pendulum that manipulating the post-synaptic gain of predictive neurons reproduced behaviors observed in schizophrenia and control populations. Such a difference in the balance of information flow could have for instance a genetic origin in the expression of this gain and vicariously in the behavior of this population. Importantly, such a method thus allows to perform quantitative predictions: Such applications of computational neuroscience seem particularly relevant for a better understanding of the diversity of behaviors in the human population (see for instance [@Karvelis18autistic; @Kent19]).

Outline

Illusions visuelles & hallucinations

Hallucinations: quand les illusions deviennenty réelles

Une neuro-anatomie fonctionnelle des illusions?

Cydonia Mensae (1976) Viking Orbiter image

Bressloff, P. C., Cowan, J. D., Golubitsky, M., Thomas, P. J., & Wiener, M. C. (2002). What geometric visual hallucinations tell us about the visual cortex. Neural Computation, 14(3), 473–491.

Here, we sum- marize a theory of their origin in visual cortex (area V1), based on the assumption that the form of the retino–cortical map and the architecture of V1 determine their geometry.

Using this symmetry, we show that the various patterns of activity that spontaneously emerge when V1’s spatially uniform resting state becomes unstable correspond to the form constants when transformed to the vi- sual 􏰮eld using the retino-cortical map. The results are sensitive to the detailed speci􏰮cation of the lateral connectivity and suggest that the cortical mechanisms that generate geometric visual hallucinations are closely related to those used to process edges, contours, surfaces, and textures.

Une neuro-anatomie fonctionnelle des illusions?

Cydonia Mensae (1976) Viking Orbiter image

Bressloff, P. C., Cowan, J. D., Golubitsky, M., Thomas, P. J., & Wiener, M. C. (2002). What geometric visual hallucinations tell us about the visual cortex. Neural Computation, 14(3), 473–491.

Here, we sum- marize a theory of their origin in visual cortex (area V1), based on the assumption that the form of the retino–cortical map and the architecture of V1 determine their geometry.

Une neuro-anatomie fonctionnelle des illusions?

Cydonia Mensae (1976) Viking Orbiter image

Bressloff, P. C., Cowan, J. D., Golubitsky, M., Thomas, P. J., & Wiener, M. C. (2002). What geometric visual hallucinations tell us about the visual cortex. Neural Computation, 14(3), 473–491.

Here, we sum- marize a theory of their origin in visual cortex (area V1), based on the assumption that the form of the retino–cortical map and the architecture of V1 determine their geometry.

Une neuro-anatomie fonctionnelle des illusions?

Cydonia Mensae (1976) Viking Orbiter image

Bressloff, P. C., Cowan, J. D., Golubitsky, M., Thomas, P. J., & Wiener, M. C. (2002). What geometric visual hallucinations tell us about the visual cortex. Neural Computation, 14(3), 473–491.

Here, we sum- marize a theory of their origin in visual cortex (area V1), based on the assumption that the form of the retino–cortical map and the architecture of V1 determine their geometry.

Une neuro-anatomie fonctionnelle des illusions?

Jancke, Chavane, Naaman and Grinvald (2004) Nature

Une neuro-anatomie fonctionnelle des illusions?

Jancke, Chavane, Naaman and Grinvald (2004) Nature

In a recent study [@Chemla19], we used VSDI to record the activity of the primary visual cortex (V1) of awake macaque monkeys. Is there any difference between the response to the single dot and that to the two dots? Indeed, VSD recordings allow to record the activity of populations of V1 neurons which are approximately at the scale of a cortical column. Additionally, the recorded response is rapid enough to capture the dynamics of the lrAM stimulus. Recordings show that as the evoked activity of the second stimulus reaches V1, a cortical suppressive wave propagates toward the retinotopic wave evoked by the first dot. This was put in evidence by statistically comparing the response of the brain to the response of the two dots in isolation. In particular, we found that thanks to this suppressive wave, the activity for the brain stimulus was more precise, suggesting that such suppressive wave could serve as predictive processing step to be read-out in upstream cortical areas.

In particular, we found that the activity that we recorded fitted well with a mean-field model using a dynamical gain control. Qualitatively, this model reproduced the propagation of activity on the cortex. Importantly, this model allowed to show that the observed activity was best fitted when the speed of lateral connections within the mean-field was about 1 m/s, a propagation speed which is of the order of that measured for intra-cortical connections in the primary visual cortex (for a review, see [@Muller18]). A more functional (probabilistic) model also showed that the cortical suppressive wave allowed to disambiguate the stimulus by explaining away (that is, suppressing) ambiguous alternatives. As a consequence, (1) lateral interactions are key to generate traveling waves on the surface of the cortex and (2) these waves help disambiguate the input stimulus. This corresponds to the implementation of a predictive process using an a priori knowledge of smoothly-moving visual objects.

Des illusions aux hallucinations visuelles:
une porte sur la perception

Laurent Perrinet

JNLF 2019, 18/4/2019

Acknowledgements:

Rick Adams and Karl Friston @ UCL - Wellcome Trust Centre for Neuroimaging
Jean-Bernard Damasse, Laurent Madelain and Anna Montagnini - ANR REM
Frédéric Chavane - INT

This work was supported by ANR project "Horizontal-V1" N° ANR-17-CE37-0006.

https://laurentperrinet.github.io/talk/2019-04-18-jnlf

Des illusions aux hallucinations visuelles: une porte sur la perception

Laurent Perrinet

JNLF 2019, 18/4/2019

Acknowledgements:

Des illusions aux hallucinations visuelles: une porte sur la perception

Laurent Perrinet

JNLF 2019, 18/4/2019

Acknowledgements:

Paréidolie

Outline

Illusions visuelles & hallucinations

Hallucinations: quand les illusions deviennenty réelles

Une neuro-anatomie fonctionnelle des illusions?

Hallucinations: quand les illusions deviennenty réelles

Outline

Illusions visuelles & hallucinations

Hallucinations: quand les illusions deviennenty réelles

Une neuro-anatomie fonctionnelle des illusions?

Une neuro-anatomie fonctionnelle des illusions?

Une neuro-anatomie fonctionnelle des illusions?

Une neuro-anatomie fonctionnelle des illusions?

Une neuro-anatomie fonctionnelle des illusions?

Une neuro-anatomie fonctionnelle des illusions?

Une neuro-anatomie fonctionnelle des illusions?

Des illusions aux hallucinations visuelles: une porte sur la perception

Laurent Perrinet

JNLF 2019, 18/4/2019

Acknowledgements:

Des illusions aux hallucinations visuelles:
une porte sur la perception

Des illusions aux hallucinations visuelles:
une porte sur la perception

Des illusions aux hallucinations visuelles:
une porte sur la perception